Multiprocessor system having floating executive control



March 19, 1968 R. c. RICHMOND ET AL 3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CGNTROL 18 Sheets-SheetFiled March 19, 1965 March 19, 1968 R. C. RICHMOND ET AL 3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL la Sheets-SheetFiled March 19, 1965 March 19, 1968 R. c. RICHMOND ET AI- 3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL Filed March 19,1965 /5I/ /JZ 13 Sheets-Sheet 3 March 19, 1968 R. c. RICHMOND ET Al-3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL 123 Sheets-Sheetf,

Filed March 19, 1965 March 19, 1968 Filed March 19, 1965 R. C. RICHMONDET AI- MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL PMK/00@Iza-7.

18 Shee tS-Shee t Oy/Of fffzff.

March 19, 1968 R. c:A RICHMOND ET AL 3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL Filed March 19,1965 18 Sheets-Sheet March 19, 1968 R. c. RICHMOND ET AL 3,374,465

MULTIFROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL 123 Sheets-SheetFiled March 19, 1965 www g MQ

March 19, 1968 R. c. RlcHMoND ETAI- 3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CGNTROL Filed March 19,1965 18 Sheets-Sheet 5 0545/7 Il /74 205 /ZJ fao a zoo E /O y 0//7 E l45 /75 March 19, 1968 R. c. RICHMOND ET Al- 3,374,465

MULTIPRCESSOR SYSTEM HAVING FLOATIN EXECUTIVE CONTROL' Filed March 19,1965 18 Sheets-Sheet .A

March 19, 1968 R. c. RICHMOND ET Al- MULTIPROCESSOR SYSTEM HAVINGFLOATING EXECUTIVE CONTROL Filed March 19, 1965 MVC/064 MVC /a7 ZIM@ 500MVC /07 MyC/07 @mfc/065 18 Sheets-Sheet o Eze/ March 19, 1968 FiledMarch 19, 1965 R. C. RICHMOND ET AL MULTIPROCESSOR SYSTEM HAVINGFLOATING EXECUTIVE CONTROL 18 Sheets-Sheet 1l Eze.- /9.

CONTROL March 19, 1968 R. c. RICHMOND ET AL MULTIPROCESSOR SYSTEM HAVINGFLOATING EXECUTIVE 18 Sheets-Sheet Filed March 19, 1965 Nmxhqu ESQ@March 19, 1968 R. c. RICHMOND ET AL 3,374,465

EXECUTIVE CONTROL MULTIPHOCESSOR SYSTEM HAVING FLOATING 123 Sheets-Sheet5 W W W W W v W W W W W W W W W W W W W W L W W W W W W W SQ V/ /W W Wbh. W W W W W .W NQ Fad. W W W W W n W W W n W Y W d W d W Y QuRm/.WJVMWWMMWMWMWMWM WM M w (MWA .../Wm /WmA aWa Mm ,nk @aWmWwvwWzmaWai/.m IW Z W W NW Wllr QH! S w W W Wwn. WF Wb? um. mvkd W W W Mk.W W. W Wmh.\ mwN W Si :i :im W Si m\ W W BQWQQQ mum .WNQQW RGWQQS Si.WNQQW En uns Ek wvl March 19, 1968 R. c. RICHMOND ET AL 3,374,465

MULTIEROCESSOR SYSTEM HAVING FLOA'IING EXECUTIVE CONTROL Filed March 19,1965 18 Sheets-Sheet 14 ZIJ afa:

afa!

50a Zig afa! p Il;

a ,3 ze; V a

of /v March 19, 1968 R. c. mcs-MONO ET Al- 3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL Filed March 19.1965 18 Sheets-Sheet 1.5., WL L MMMLJA f 5747; aaa /a /62 as /70 /7/ /rzan: aaa aa/ aa: au ma March 19, 1968 R. C. RICHMOND ET AL 3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL Filed March 19,1965 18 Sheets-Sheet 16 pcggirw Vf/7p ,didl/fai V F/@ K Jf/-A/Msrzacr/@A/ [pi/I/]a}a|/}o1p]plpI/]a|a|c|/Vidal/530 Pa-.32. .55.: l I 4[/alala] 2 olaf/pple] @L w w Zag feg f L NF fz L/ I 4 l/Ia[a[ol .aHalal/lalala] ser @7.4

@7A/fz 745K fea vicar/V;

oar/Ms Ilz-:255. A?

J' zac/:M

la can ,6 Y y a 45 i 5 /7 T1970 a /7 f i I V22 a f z 5 4 5 /7 M55 726,-.an

March 19, 1968 R. c. RICHMOND ET Al- 3,374,465

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL Eze.- 39.

March 19, 1968 R. c. RICHMOND ET AL 3,374,455

MULTIPROCESSOR SYSTEM HAVING FLOATING EXECUTIVE CONTROL Filed March 19,1965 18 Sheets-Sheet l5 United States Patent Olice Patented Mar. 19,1968 3.374.465 MULTIROCESSQR SYSTEM HAVING FLOATING EXECUTIVE CONTROLRichard C. Richmond and Jack J. Pariser, Orange, and

Thomas A. Connolly, Hacienda Heights, Calif., assignors to HughesAircraft Company, Culver City, Calif., a corporation of Delaware FiledMar. 19. 1965. Ser. No. 441,189 20 Claims. (Cl. S40-172.5)

ABSTRACT OF THE DISCLOSURE A multiprocessor computer system comprisingan executive control logic circuit and an executive function logiccircuit in each processor with the executive control logic circuitsfunctioning as program indicators. Any of the executive control logiccircuits may be set as a result of the processors set instruction or ofany processors reset instruction, The executive function logic circuitiS maintained in a set state during an executive routine or an interruptroutine. The system provides an executive or type of control circuit sothat the computers operation may be continued in event of a failure of aportion of the processors and to ensure that one and only one processorhas executive control during any given time period.

This invention relates to multiprocessor computer systems andparticularly to a system utilizing a plurality of processors in whicheach processor continually has an equal opportunity to exerciseexecutive control of the overall computing operation.

Multiprocessor computer systems are advantageous for providingrelatively high computational speeds such as in real time processingsystems, or in any system where a maximum amount of processing isdesired to be performed for any period of time In real time operation,for example, multiprocessor systems are desirably arranged with separategroups of elements or modules for obtaining system survivability in theevent of a unit failure. A further advantage of multiprocessors inmodular arrangements is that additional processor and memory modules maybe added or removed from an installation as required by changes in thejob requirements. Conventional multiprocessor systems operate in amaster-slave control relationship in which a predetermined processoralways has executive control or in an independent manner in which eachcomputer including a processor and memory solves its set of problems andperiodically exchanges information with other computers. Multiprocessorsystems in master-slave configurations suffer from inefficiencies andmay be completely disabled by the failure of the master processor. In anarrangement in which the master processor is capable of performing onlyexecutive control functions, the master processor may be designed to befully loaded for one program configuration of specic slave processor orprocessors. However, this system will not be efficient in any processorand program configuration which requires less executive control.Regardless of the efficiency of the master processor, the total systemeihciency of the master-slave system may be reduced as a result ofprogram or system configuration changes, because the slave processor orprocessors may be required to wait for the master processor to `providethe next task assignment. Similarly, a system in which each processoroperates independently of the other with periodic exchanges of data isineticient because of the time required to exchange data and becausewaiting periods may be required before data is available to perform adependent program. Another disadvantage of a plurality of independentlyoperating processors is that even though a program may be selected tominimize waiting or idle times, interrupt times such as duringperformance of a routine in response to a request from an inputoutputunit may result in substantially long waiting periods of someprocessors.

It is therefore an object of this invention to provide a multiprocessorcomputer system that operates with a high degree of overall efficiency.

It is another object of this invention to provide a multiprocessorsystem in which the task assignment function may be transferred betweendifferent processors by an exclusive or type control operation.

It is still another object of this invention to provide a processingsystem in which each of a plurality of data handling units have an equalopportunity to exercise executive control of the overall operation butin which only one data handling unit exercises executive control at anyone time.

It is a further object of this invention to provide a multiprocessorcomputer system in which the executive control function may becontinually transferred between processor units with the processor unithaving executive control responding to interrupt and selectedinputoutput operations.

lt is a still further object of this invention to provide amultiprocessor system in which system operation may continue uponfailure of a portion of the processors.

It is a further object of this invention to provide a system in whichmultiprocessor or single processor operation may be selected.

it is a further object of this invention to provide a multiprocessorsystem in which indicators control the transfer of the executivefunction.

lt is a further object of this invention to provide a multiprocessorsystem in which the program interrupt operation and the executivefunction utilize common control elements.

It is a further object of this invention to provide au improvedmultiprocessor system utilizing n plurality of memory banks.

Briefly the multiprocessor system having floating executive control inaccordance with the principles of the invention includes a plurality ofprocessors each capable of accesjing a memory system which may be amultiple bank memory in some arrangement in accordance with theprinciples of the invention. Each processor has executive controlindicators which muy include an executive control flip llop and anexecutive function iiip flop. Both flip liops may be set as a result ofu computer instruction and the last instruction of a job routine mayrequest both executive control and executive function which is grantedin the requesting processor if the other processor is not performing anexecutive function. Setting the executive control liip iiop of oneprocessor resets the control flip liop of the other processors toprovide an exclusive or type executive control. The executive controlflip flop may be set as a result of its processors set instruction or asa result of another processor-s reset instruction. Sct instructions areobeyed by the executive control liip iiop only when no other processoris engaged in an executive task as indicated by the executive functionindicator and the set instructions are obeyed by the executive functionliip flop only when that processors executive control indicator is set.in response to a failure condition of any processor, that processorforces another processor to aC- ccpt the executive control. Bolh theinterrupt controls and a portion ofthe input-output operation arecontrolled by the processor' having the executive responsibility. Thesystem in accordance with the invention allows a new program to beutilized upon failure of a portion of the processors.

The novel features of the invention, as well as the invention itself,both as to its organization and method of operation, will best beunderstood from the accompanying description, taken in connection withthe accompanying drawings, in which like reference characters refer tolike parts, and in which:

FIG. l is a schematic block diagram of the multiprocessor, iioatingexecutive control system in accordance with the principles of theinvention;

FIG. 2 is a schematic block diagram showing the banked memory system ofFIG. l in further detail;

FIG. 3 is a schematic circuit diagram of a typical NAND (negative and)gate for explaining an example of one type of logical structure that maybe utilized in the system of the invention',

FIG. 4 is a schematic block diagram of a control iiip flop forexplaining an example of one type of Hip op that may be utilized in thesystem of the invention;

FIG. 5 is a schematic block diagram of the address register of FIG. 2for one of the memory banks and typical for the other memory bank;

FIG. 6 is a schematic block diagram of the memory phase counter ip flopsfor a first one of the memory banks and typical for the other memorybank of FIG. 2;

FIG. 7 is a schematic logical diagram of portions of the controlcircuits of FIG. 2;

FIG. 8 is a schematic logical block diagram showing the write memorycycle flip op utilized in the control network of a first one of thememory banks of FIG. 2 and typical of similar llip ops utilized in theother memory bank;

FIG. 9 is a schematic logical block diagram of the input-output cycleflip hops utilized in the control network of a rst one ofthe memorybanks of FIG. 2 and typical of similar ip ops utilized in the othermemory bank;

FIG. l() is a schematic logical block diagram of `the arithmetic unitcycle ip flops utilized in the control network of the memory banks ofFIG. 2;

FIG. l1 is a logical block diagram showing a portion of the feedbackunits of FIG. 2;

FIG. 12 is a logical block diagram showing a portion of the dataselection networks of FIG. 2 for gating data to the arithmetic unit bus;

FIG. 13 is a logical block diagram showing a portion of the dataselection networks of FIG. 2` for gating data to the input-output unitbus;

FIG. 14 is a logical block diagram of a portion of the address registerof each memory bank of FIG. 2 for gating the address from the arithmeticunit and input-output unit sources;

FIG. l5 is a schematic logical block diagram ofthe data register in afirst one of the memory banks and typical of the data register in theother memory bank of FIG. 2;

FIG. 16 is a logical diagram showing gates for developing controlsignals to be utilized in the data registers of thc memory banks of FIG.l;

FIG. 17 is a schematic logical diagram showing the selection networks ofFIG. 2 for each of the memory banks;

FIG. I8 is a schematic logical diagram of gates for developing thearithmetic unit memory request signals in the processors of FIG. 1;

FIG. 19 is a schematic circuit and block diagram of the executivefunction indicator Hip op that may be utilized in each of the processorsof FIG. 1 for controlling the iloating executive operation in accordancewith the invention;

FIG. 2t) is a schematic block and circuit diagram of the executivecontrol indicator ip flop that may be utilized in cach of the processorsof FIG. 1 for controlling the floating executive operation in accordancewith the invention;

FIGS. 21 and 22 are schematic circuit and block diagrams of the programcontrol unit sequencer ip flops utilized in each of the processors ofFIG. l for controiling the sequences of operation during performance ofprogram instructions;

FIG. 23 is a schematic circuit and block diagram of the control logicfor transferring plus "one" into the adder in each processor of FIG. l;

FIG. 24 is a schematic circuit diagram of the skip on indicator switchesutilized in each of the processors of FIG. 1 to pass into an executiveroutine when executive control is obtained;

FIG. 25 is a schematic circuit diagram of the gating structure in eachprocessor for controlling the transfer of the contents of the programcounter of FIG. l into the adder during a skip on indicator instruction;

FIG. 26 is a schematic circuit and block diagram of a typical interruptliip flop that may be utilized in each of the processors of FIG. l;

FIG. 27 is a schematic circuit diagram of a typical gating strcture usedin each processor of FIG. l for developing priority interrupt signals;

FIG. 28 is a schematic circuit diagram of a typical gating structure fordeveloping an interrupt priority signal to be utilized in the circuit ofFIG. 27 in the processor having executive control;

FIG. 29 is a schematic circuit and block diagram of the interruptcontrol ip op X11 utilized in each of the processors of FIG. l;

FIG. 30 is a schematic circuit and block diagram of the clock updatecontrol circuit utilized in each of the processors of FIG. l;

FIG. 3l is a schematic diagram showing the format of the words that maybe utilized in the system of FIG. l;

FIG. 32 is a schematic diagram showing the transfer of data during setindicator and reset indicator instructions in each ofthe processors ofFIG. l;

FIG. 33 is a schematic diagram of waveforms showing voitage as afunction of time for further explaining the operation of indicator setand reset and skip on indicator instructions in cach of the processorsof FIG. l',

FIG. 34 is a schematic diagram of waveforms showing voltage as afunction of time for explaining the operation of the interrupt routineutilized in the processors of FIG. l in accordance with the principlesof the invention;

FIG. 35 is a schematic ow diagram of the operation of the processors ofFIG. 1 to obtain executive control in accordance with the principlesofthe invention;

FIG. 36 is a schematic fIow diagram showing a typical executive routinethat may be performed by a processor obtaining executive control;

FIG. 37 is a logical state table for explaining the relations oftheexecutive control and executive function indicators of the system ofFIG. l when utilizing two processors;

FIG. 38 is a schematic block diagram for explaining the oating executiveoperation with three interconnected processors in accordance with theprinciples of the invention; and

FIG. 39 is a schematic diagram of program processing time for furtherexplaining the operation of the multi processor system of FIG. 1.

Referring first to FIG. 1, the multiprocessor system in accordance withthe principles of the invention includes a memory system 9 having memorybanks 10 and 12, as well as additional banks in some arrangements inaccordance with the invention as indicated by a dotted bank 13. Thememory bank l0 and 12 may include magnetic memory array units 18 and 19,each storing information in magnetic cores, thin films, magnetic wiresor other suitable storage arrangements. It is to be noted that theprinciples of the invention are applicable to processors uti lizingother memory arrangements and the bank system is illustrated as oneexample of a type that may be utilized in accordance with the invention.As is wel] known in the art, the memory array units 18 and 19 have aplurality of cells or word positions at which either instruction

